On the importance and incorporation of additional knowledge in cluster analysis

Analysis of data without labels is commonly subject to scrutiny by unsupervised machine learning techniques. Although abundant expert knowledge exists in many areas where unlabelled data is examined, frequently such knowledge is not incorporated into automatic analysis. Semi-supervised learning allows for the incorporation of additional knowledge with the help of labels or constraints. However it is the field of supervised learning and the recently proposed advanced paradigm of learning using privileged information that provides an intriguing concept of incorporating special type of additional knowledge. In this thesis we explore the question of importance and incorporation of such additional knowledge within unsupervised learning. Our analysis is performed from four different viewpoints, namely anomaly detection, cluster interpretation, visualisation and identification. The functionality of signal fusion and low-level pattern recognition in the human immune system is our inspiration. A more practical set of immunology derived techniques is developed, allowing for the fusion of additional information for improved anomaly detection (UR-STD), cluster interpretation and visualisation (StOrM). The success of these techniques within computer security and process behaviour scenarios encouraged further exploration of additional knowledge incorporation at a more general level. Adoption of the advanced supervised learning paradigm for the unsupervised setting instigates investigation into the difference between privileged and technical data. By means of our proposed aRi-MAX method stability of the K-Means algorithm is improved and identification of the best clustering solution is achieved on an artificial dataset. Subsequently an information theoretic dot product based algorithm called P-Dot is proposed. This method has the ability to utilize a wide variety of clustering techniques, individually or in combination, while fusing privileged and technical data for improved clustering. Application of this P-Dot method to the task of digit recognition confirms our previous findings in a real-world scenario. Finally the nature of privileged information is investigated more formally and its association with the notion of data generators is revealed. Experiments using standard machine learning datasets uncover the hypothetical existence of disparate data generators, which can provide benefits for unsupervised learning when treated uniquely, i.e. equally. The incorporation of additional knowledge within cluster analysis can be beneficial, especially when generators are identified and treated separately and with regards to their nature.